Constraints for the Estimation of Displacement Vector Fields From Image Sequences

Smoothness constraints have been used to f a c i l i tate the estimation of displacement vector f i e lds . Differing from HORN and SCHUNCK 81 who employ a general smoothness requirement, this contribution reports an analysis of an "oriented smoothness" requirement: a change in the displacement vector f i e ld is only constrained in the direction perpendicular to the characteristic gray value variation based on which the displacement vector is es t i mated. An i terat ive solution for the resulting system of nonlinear part ial d i f ferent ia l equations is developed. It is shown how this system of equations relates to the one derived by HORN and SCHUNCK 81. 2. "ORIENTED SMOOTHNESS" REQUIREMENTS Several approaches have been investigated in order to cope especially with d i f f i cu l t y ( i ) . YACHIDA 82 assumes that a displacement vector can be es t i mated at prominent points. He propagates such es t i mates into neighboring areas with large gray value gradients, based on the method of HORN and SCHUNCK I—/ They employ the variational calculus to transform equation (2) into a system of second order part ial d i f ferent ia l equations for the two components u(X) and v(X). Using a part icular approximation on a 3x3 pixel grid for the Laplacian of u and v, this system of two coupled part ial d i f ferent ia l equations has been converted into a sparse system of l inear equations with two unknowns for each pixel in the image. An i tera t ive solution of this large system of equations has been employed in order to study various examples. Two aspects of this approach let it appear espec ia l l y at t ract ive. F i rs t , a l l hypotheses are in troduced exp l i c i t l y into a rigorous mathematical framework. Second, well-known mathematical methods are employed to derive the solution so that the consequences of the various hypotheses and approximations can be studied in de ta i l . One can recognize three sources of d i f f i cu l t i es in the approach of HORN and SCHUNCK 81: i) The smoothness requirement is applied indiscriminately across a l l gray value edges despite the fact that such edges might separate image regions with discontinuous displacement vector f ie lds (see SCHUNCK and HORN 81). i i ) No advantage is taken of the fact that certain nonlinear gray value variations may constrain possible displacement vectors much stronger than image areas with smoothly varying gray values or straight l ine gray value